The present invention relates generally to a sensing device for capturing interactions with and movement relative to a surface, and more particularly, such interactions with and movement relative to an electronic ink surface.
Pen-like devices have been described which mark a surface while simultaneously capturing their own movement relative to the surface. Some such devices sense their own movement using embedded accelerometers. Others sense their own movement by detecting information which is machine-readable encoded on the surface. In this latter category, see for example U.S. Pat. No. 5,477,012, U.S. Pat. No. 5,652,412 and PCT application W099/50787.
Since these pen-like devices typically utilize conventional marking means such as inking nibs, the surface media being marked are not generally re-usable.
In accordance with one aspect of the present invention, there is provided a sensing device for use with a surface having coded data and electronic ink disposed on the surface, the coded data being indicative of an identity of a region associated with the surface, the sensing device including:
a region identity generator configured to generate region identity data indicative of the identity of the region using the coded data; and
a transmitter configured to transmit the region identity data to a computer system.
Preferably, the sensing device further includes motion sensing means configured to generate movement data indicative of movement of the sensing device relative to the region. The motion sensing means is preferably configured to generate the movement data using at least some of the coded data.
The coded data is preferably indicative of a plurality of reference points of the region, the motion sensing means being configured to generate the movement data as the sensing device moves relative to at least one of the reference points. The coded data may include periodic elements, the motion sensing means being configured to generate the movement data as the sensing device moves relative to at least one of the periodic elements.
Preferably, the motion sensing means is configured to sample the position of the sensing device relative to the at least one reference point, thereby to generate the movement data. Alternatively, or in addition, the motion sensing means is configured to sample the position of the sensing device relative to the at least one periodic element, thereby to generate the movement data.
Preferably, the motion sensing means includes at least one acceleration sensing means, the acceleration sensing means being configured to sense acceleration of the sensing device as the sensing device moves relative to the region, the motion sensing means being configured to generate the movement data by periodically sampling the acceleration. The acceleration sensing means may be configured to sense at least two substantially orthogonal components of acceleration.
The coded data can be recorded on the surface separately or when the surface is being manufactured. The surface may be any surface including paper, plastics or any other solid material. The surface may be part of another item. For example, it could be part of furniture, part of a wall, part of any object. Preferably, the surface is a page, preferably a page of paper, and may be part of a document.
The sensing device need not actually be in contact with the surface. It may be in contact with the surface, but it may also remain out of contact with the surface and merely pass over the surface.
Information on the position over the surface of the apparatus is preferably transmitted by a transmitter to a computer system, where it can be interpreted. If a person is making a drawing on a sheet of paper on which the coded data is recorded, therefore, the computer system can interpret the positional information provided by the device so that data indicative of the drawing can be stored in the computer and reproduced as, for example, a printed page. Alternatively, the positional information may be used to control the computer system. The preferred embodiment which utilizes the sensing device in conjunction with a computer system will be described later.
Preferably, the device further includes a state changing device for changing the state of the electronic ink. In this way the surface can be visibly marked or previously applied marks can be erased by changing the state of the electronic ink as, the device moves relative to the surface.
Preferably, the state changing device is activatable and deactivatable.
The device preferably further includes a force sensor adapted to activate or deactivate the state changing device when the force between the device and the surface exceeds a predetermined value.
Preferably, the device further includes means responsive to a marker associated with the surface to enable or disable the state changing device in response to detection of the marker.
Preferably, the state changing device includes an electric field generator.
Preferably, the state changing device includes a first state changing device for changing the state of the electronic ink to a first state, and a second state changing device for changing the state of the electronic ink to a second state.
Preferably, the device is elongate, the first state changing device is disposed adjacent a first longitudinal end of the sensing device and the second state changing device is disposed adjacent a second opposite longitudinal end of the sensing device.
Preferably, the coded data includes a plurality of tags, each tag being indicative of an identity of a region within which the tag lies, and each tag being indicative of a reference point of the region, the region being associated with the surface, and the reference point being indicative of the position of the tag relative to the region. Alternatively, or in addition, each tag is indicative of an identity of a region within which the tag lies, and each tag includes at least one periodic element.
In accordance with an alternative aspect of the present invention, there is provided a system for capturing information applied freehand, the system including a sensing device in accordance with the first aspect of the present invention and a substrate including a surface having electronic ink associated with the surface and coded data associated with the surface.
Preferably, the surface is provided with electronic ink and color visible ink which is not electronic ink. The substrate is preferably paper.
A region may include a plurality of tags. A region may be, for example, a single, page or a specific area on a surface. Preferably, each of the tags in a region is indicative of region data which identifies that region. Where the region is a page, therefore, each one of a plurality of tags on that page is indicative of position data identifying the absolute position of the identifier on that page and also region data which identifies the age. Preferably, the detection means of the device is arranged to capture the region data and may be passed on to a computer system. This enables, for example, a computer system to identify a page so that, for example, a computer system xe2x80x98knowsxe2x80x99 the exact document which is being written or drawn on.
As discussed above, the device need not be in contact with the surface. Where it is intended to be used at least sometimes in contact with a surface, the device may include a force sensor for sensing the force of the device applied to the surface. Preferably, the device includes a force data producing means for producing force data which provides information on the force. The information can preferably be utilized in at least two ways.
Firstly, it can be used to determine when the device is first applied to the surface and when it leaves the surface, with motion between force application and removal being defined as a xe2x80x98strokexe2x80x99 in freehand. The force data information can be time stamped.
Secondly, it can be used to detect, on a substantially continuous basis, force variations when the device is applied to the surface. This information may be utilized to characterize a signature of a user, for example, or may be used for a further control for a computing system.
The device is preferably a separate implement containing the appropriate means as discussed above. It may be any shape but it is preferably in the form of a stylus or pen.
The device is preferably intended for interaction with a computer system that can be controlled and can interpret the freehand information (whether drawing or writing) applied by a user via the device. Preferably, the device is arranged to provide device identification information which uniquely identifies the device. The computer system may therefore use this to identify the device.
In accordance with a further alternative aspect of the present invention, there is provided a substrate including a surface having electronic ink associated with the surface and coded data associated with the surface, the coded data being indicative of an identity of a region associated with the surface.